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Abstract. The concept of entropy in nonequilibrium macroscopic systems is investigated 
in the light of an extended equation of motion for the density matrix obtained in a previous 
study. It is found that a time-dependent information entropy can be defined unambigu- 
ously, but it is the time derivative or entropy production that governs ongoing processes 
in these systems. The differences in physical interpretation and thermodynamic role of 
entropy in equilibrium and nonequilibrium systems is emphasized and the observable as- 
pects of entropy production are noted. A basis for nonequilibrium thermodynamics is also 
outlined. 

1. Introduction 

The empirical statement of the Second Law of thermodynamics by Clausius (1865) is 

S(initial) < S(final) , (1) 

where S is the total entropy of everything taking part in the process under consideration, 
and the entropy for a single closed system is defined to within an additive constant by 



where C(T) is a heat capacity. The integral in (2) is to be taken over a reversible path, 
a locus of thermal equilibrium states connecting the macroscopic states 1 and 2, which is 
necessary because the absolute temperature T is not defined for other than equilibrium 
states; dQ represents the net thermal energy added to or taken from the system in the 
process. As a consequence, entropy is defined in classical thermodynamics only for states 
of thermal equilibrium. Equation (1) states that in the change from one state of thermal 
equilibrium to another along a reversible path the total entropy of all bodies involved 
cannot decrease; if it increases, the process is irreversible. That is, the integral provides a 
lower bound on the change in entropy. This phenomenological entropy is to be found from 
experimental measurements with calorimeters and thermometers, so that by construction 
it is a function only of the macroscopic parameters defining the macroscopic state of a 
system, S(V,T, N), say, where V and iV are the system volume and particle number, 
respectively. It makes no reference to microscopic variables or probabilities, nor can any 
explicit time dependence be justified in the context of classical thermodynamics. Equation 
(1) is a statement of macroscopic phenomenology that cannot be proved true solely as 
a consequence of the microscopic dynamical laws of physics, as appreciated already by 
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Boltzmann (1895): "The Second Law can never be proved mathematically by means of 
the equations of dynamics alone." (Nor, for that matter, can the First Law!) 

Theoretical definitions of entropy were first given in the context of statistical me- 
chanics by Boltzmann and Gibbs, and these efforts culminated in the formal definition 
of equilibrium ultimately given by Gibbs (1902) in terms of his variational principle. In 
Part I (Grandy, 2003, preceding paper^) we observed that the latter is a special case of a 
more general principle of maximum information entropy (PME), and in equilibrium it is 
that maximum subject to macroscopic constraints that is identified with the experimental 
entropy of (2). One of the dominant concerns in statistical mechanics and thermody- 
namics has long been that of extending these notions unambiguously to nonequilibrium 
phenomena and irreversible processes, and it is that issue we shall address in this work. 

How is one to define a time-dependent S(t) for nonequilibrium states? Do there even 
exist sensible physical definitions of experimental and theoretical 'entropy' analogous to 
those describing equilibrium states? Other than S(t) and the density matrix p(t), what 
other key parameters might be essential to a complete description of nonequilibrium? 
These and other questions have been debated, sometimes heatedly, for over a century 
without any broad concensus having been reached; perhaps a first step toward clarifying 
the issue should be to understand the source of the differences of opinion that lead to so 
many different points of view. One problem, of course, is a lack of experimental guidance 
in determining those features of the phenomena that are really of fundamental importance, 
and associated with this has been the necessary restriction of theories to linear departures 
from equilibrium, owing to enormous calculational difficulties with the appropriate non- 
linear forms. What happens, then, is that many theoretical descriptions of nonequilibrium 
systems tend to predict similar results in the linear domain and there is little to distinguish 
any fundamental differences that get to deeper matters. 

In view of these obstacles it may be useful to look at the problem from a different 
perspective. Such sharp disagreements would seem to arise from different hidden premises 
in various approaches to nonequilibrium statistical mechanics, and we suggest here that 
these have much to do with differing views of the underlying probability theory and its 
precise role. We initiated an examination of this point in I, which culminated in the 
expression (1-40) as the appropriate form of the equation of motion for the density matrix. 
Our purpose here is to apply the implications of that result to a further study of time 
varying macroscopic systems. 

As a preliminary step it might be helpful to note some overall features of entropy and 
nonequilibrium processes that have to be considered in any approach to the problem of gen- 
eralizing S. Suppose we prepare a system in a nonequilibrium state by applying an external 
force of some kind that substantially perturbs the equilibrium system, possibly increasing 
its energy and adding matter to it, for example. This state is defined by removing the 
external source at time t = to, and at that instant it is described by a density matrix p(to). 
Whether or not we can define a physical entropy at that time, we can certainly compute 
the information entropy of that nonequilibrium state as S/(to) = — kTr[p(to) lnp(£ )]- Be- 
cause the system is now isolated, pit) can only evolve from p(to) by unitary transformation 



t Equation (n) of that paper will be denoted here by (I-n). 
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and Si remains constant into the future. ^ What happens next? 

At the cutoff t = to the entropy Si (to) refers only to the nonequilibrium state at 
that time. In the absence of any other external influences we expect the system to relax 
into a new equilibrium state, for no other reason than it is the one that can be realized 
in the overwhelmingly greatest number of ways subject to the appropriate macroscopic 
constraints. The interesting thing is that those constraints are already fixed once the ex- 
ternal sources are removed, so that the total energy, particle number, volume, etc. at to 
are now determined for t > to- The entropy of the final equilibrium state is definitely not 
Si (to), but it is in principle known well before equilibrium is reached: it is the maximum 
of the information entropy subject to constraints provided by the values of those thermo- 
dynamic variables at t = to- We may or may not know these values, of course, although 
a proper theory might predict them; but once the final macrostate is established they can 
be measured and a new pf calculated by means of the PME, and hence a new entropy 
predicted for comparison with the experimental form of Clausius; indeed, in equilibrium 
the Clausius entropy (2) is an upper bound for Si- Thus, in this relaxation mode we do 
not see a nice, continuous, monotonically increasing entropy function that can be followed 
into the equilibrium state; but that's not too surprising, given that we know p(to) cannot 
evolve unitarly into pf. There remains a significant dynamical evolution during this re- 
laxation period, but it is primarily on a microscopic level; its macroscopic manifestation 
is to be found in the relaxation time, and the possible observation of decaying currents. 
One thing we might compute and measure in this mode is that relaxation time, which does 
not have an immediate or necessary connection with entropy. (There may, however, exist 
a 'relaxation entropy' associated with the redistribution of energy during the relaxation 
period, say.) 

Ironically, the equilibrium state described so well by classical thermodynamics is essen- 
tially a dead end; it is a singular limit in the sense discussed by Berry (2002). Equilibrium 
is actually a very special, ideal state, for most systems are not usually in equilibrium, at 
least not completely. As external influences, and therefore time variations, become smaller, 
the system still remains in a nonequilibrium state evolving in time. In the limit there is 
a discontinuous qualitative change in the macroscopic system and its description. That is, 
there is no longer either an 'arrow of time' or a past history t, and the main role of the 
theory is to compare neighboring states of thermal equilibrium without regard for how 
those states might have been prepared. 

It has long been understood that entropy is not a property of a physical system per 
se, but of the thermodynamic system describing it, and the latter is defined by the macro- 
scopic constraints imposed. The above remarks, however, lead us to view entropy more as 
a property of the macrostate, or of the processes taking place in a system. In the equi- 
librium state these distinctions are blurred, because the thermodynamic system and the 
macrostate appear to be one and the same thing and there are no time-dependent macro- 
scopic processes. It will be our goals in the following paragraphs to clarify these comments, 

t Because there are numerous 'entropies' denned in different contexts, we shall denote the exper- 
imental equilibrium entropy of Clausius as S without further embellishments, such as subscripts, 
t Requiring the equilibrium system to have no 'memory' of its past precludes 'mysterious' effects 
such as those caused by spin echos. 
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as well as to provide both an unambiguous definition of entropy in nonequilibrium systems, 
and to understand the possibly very different roles that the entropy concept plays in the 
two states. 



2. Some Preliminary Extensions of the Equilibrium Theory 

In I we briefly outlined the variational method of constructing an initial density ma- 
trix that could then evolve in time via the appropriate equations of motion. A principal 
application of that construction is to equilibrium systems, in which case the quantum form 
of (1-14) becomes 

Po = \e~^ H , Z{(5) = Tre-^ , (3) 

where H is the system Hamiltonian and (3 = (/cT) _1 . But, if there is no restriction to 
constants of the motion, the resulting state described by (3) could just as well be one 
of nonequilibrium based on information at some particular time. Data given only at a 
single point in space and time, however, can hardly serve to characterize a system whose 
properties are varying over a space-time region, so a first generalization of the technique 
is to information available over such regions. Thus, the main task in this scenario is to 
gather information that varies in both space and time and incorporate it into a density 
matrix describing a nonequilibrium state. Given an arbitrary but definite thermokinetic 
history, we can look for what general behavior of the system can be deduced from just 
this. The essential aspects of this approach were first expounded by Jaynes (1963, 1967, 
1979). 

To illustrate the method of information gathering, consider a system with a fixed time- 
independent Hamiltonian and suppose the data to be given over a space-time region -R(x, t) 
in the form of an expectation value of a Heisenberg operator F(x, t). We are reminded 
that the full equation of motion for such operators, if they are also explicitly time varying, 
is 

iUF = [F, H] + d t F . (4) 

When the data vary continuously over R their sum becomes an integral and there is a 
distinct Lagrange multiplier for each space-time point. Maximization of the entropy subject 
to the constraint provided by that information leads to a density matrix describing this 
macrostate: 

1 



p = — exp 



- / \(x,t)F(x,t)d 3 xdt 
Jr 



where 



Z[\(x,t)] = Trexp 



/ \(x,t)F(x,t)d s xdt 
Jr 



(5) 



(6) 



is now the partition functional. The Lagrange multiplier function is identified as the 
solution of the functional differential equation 

(F(x, f )) = Tr [pF(x, f )] = -^^y In Z , (x, t) G R , (7) 
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and is denned only in the region R. Note carefully that the data set denoted by (F(x, t)) 
is a numerical quantity that has been equated to an expectation value to incorporate it 
into a density matrix. Any other operator J(x, t), including J = F, is determined at any 
other space-time point (x, t) as usual by 

(J(x,t))=Tr[pJ(x,t)] = Tr[p(t)J(x)]. (8) 

That is, the system with fixed H still evolves unitarily from the initial nonequilibrium 
state (5); although p surely will no longer commute with H, its eigenvalues nevertheless 
remain unchanged. 

Inclusion of a number of operators Ff~, each with its own information-gathering region 
Rk and its own Lagrange multiplier function is straightforward. If F is actually time 
independent an equilibrium distribution of the form (3) results, and a further removal of 
spatial dependence brings us back to the canonical distribution of the original PME. But 
the full form (5) illustrates how p naturally incorporates memory effects while placing 
no restrictions on spatial or temporal scales. Nor are there any issues of retardation, for 
example, since the procedure is a matter of inference, not dynamics (at this point). 

Some further discussion is required here. The density matrix p in (5) is not a function 
of space and time; it merely provides an initial nonequilibrium distribution corresponding 
to data (F(x, £)) G R. Lack of any other information outside R — in the future, say - 
may tend to render p less and less reliable, and the quality of predictions may deteriorate. 
The maximum entropy itself is a functional of the initial values (Ffc(x, t)) E Rk and follows 
from substitution of (5) into the information entropy: 

S none(l [{(F k )}} = k\nZ[{X k }} + kJ2 [ Xk(^t)(F k (^t))d 3 xdt. (9) 

k JRk 

Although there is no obvious connection of Snoneq with the thermodynamic entropy, it 
does provide a measure of the number of microscopic states consistent with the history of 
a system over the i?fc(x, t); it might thus be interpreted as the physical entropy of the initial 
nonequilibrium state (5). If we visualize the evolution of a microstate as a path in 'phase 
space-time', then S non eq is the cross section of a tube formed by all paths by which the 
given history could have been realized, a natural extension of Boltzmann's Sb = felnW, 
where W is a measure of the set of microscopic states compatible with the macroscopic 
constraints on the system. In this sense S noneq governs the theory of irreversible processes 
in much the same way as the Lagrangian governs mechanical processes. The role of entropy 
is thus greatly expanded to describe not only the present nonequilibrium state, but also 
the recent thermokinetic history leading to that state. We begin to see that here, unlike 
the equilibrium situation, entropy is intimately related to processes. 

If the information-gathering region R is simply a time interval we arrive at the initial 
state p(to) considered in the previous section. Restriction of R to only a spatial region 
leads to a description of inhomogeneous systems. For example, specifying the particle 
number density (n(r)) throughout the system, in addition to H, constitutes a separate 
piece of data at each point in the volume, and hence requires a corresponding Lagrange 
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multiplier at each point. The distribution (3) is then replaced by 

1 



z exp 



-0H + J A(r>(r') d 3 r' 



(10) 



which reduces to the grand canonical distribution if n(r) is in fact spatially constant 
throughout V, or if only the volume integral of n(r) is specified. A similar expression is 
obtained if, rather than specifying or measuring (n(r)), the inhomogeneity is introduced 
by means of an external field coupled to n(r). In that case A(r) is given as a field strength 
and (n(r)) is to be determined; that is, A is taken as an independent variable. Extensive 
application of (10) to inhomogeneous systems is given in the review by Evans (1979). 

To this point there has been no mention of dynamic time evolution; we have only 
described how to construct a single, though arbitrary, nonequilibrium macrostate based 
on data ranging over a space-time region. A first step away from this restriction is to 
consider steady-state systems, in which there may be currents, but all variables are time 
independent. The main dynamical features of the equilibrium state are that it deals only 
with constants of the motion, among which is the density matrix itself: [H, p] = 0. These 
constraints characterize the time-invariant state of a closed system, because the vanishing 
of the commutator implies that p commutes with the time-evolution operator, so that all 
expectation values are constant in time. The time-invariant state of an open system is also 
stationary, but H almost certainly will not commute with the operators {Fk} defining that 
state, and hence not with p. Nevertheless, we can add that constraint explicitly as the 
definition of a steady-state probability distribution, and the requirement that [p, H] = 
leads to the result that only that part of Fk that is diagonal in the energy representation 
is included in pJ It is reasonably straightforward to show (e.g., Grandy, 1988) that a 
representation for the diagonal part of an operator is given by 



/" 

J — c 



F — lim / e £t d t F(^t)dt 



>o+ 

,0 

/ e£t 

J — oo 
1 f° 

= lim - / F(x.,t)dt 

r^oo T J_ T 



lim e / e £t F(x,t)dt (11) 



where the time dependence of F is determined by (4), and e > 0. The second line follows 
from an integration by parts; the third is essentially Abel's theorem and equates the 
diagonal part with a time average over the past, which is what we might expect for a 
stationary process. That is, F d is that part of F that remains constant under a unitary 
transformation generated by H. 

Consider a number of operators -Ffc(x) defining a steady-state process. Then the 
steady-state distribution p ss is simply a modification of that described by (5) and (6): 



1 

p ss = — exp 

Zj ss 



W \ k (x)Fg(x)d 3 x 



(12) 



t This prescription for stationarity was advocated earlier by Fano (1957), and has also been 
employed by Nakajima (1958) and by Kubo, et al (1985). 
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where 



Z ss [A(x)] = Trexp 



E / A fc (x)F fc d (x)d 3 x 



(13) 



We illustrate some applications of these expressions further on, but note that in their full 
nonlinear form they present formidable difficulties in calculations. 

This last caveat suggests that we first examine small departures from equilibrium, 
much in the spirit of Eqs.(I-17)-(I-19). Suppose the equilibrium distribution to be based on 
expectation values of two variables, (/) and (g), with corresponding Lagrange multipliers 
A/, X g . We also suppose that no generalized work is being done on the system, so that 
only 'heat-like' sources may operate. A small change from the equilibrium distribution can 
be characterized by small changes in the Lagrange multipliers, which in turn will induce 
small variations in the expectation values. Thus, 

w = ls?' A ' + ^S (14a) 

But from (I-13b) and (1-14)) the negatives of these derivatives are just the covariances of 
/ and g, 

_ d(f) _ d(g) 



Kf 9 = K gf = 



8Xg dX f (15) 

(fg) -(f)(9), 



so (14) reduce to the matrix equation 



Hf)\ = _(K ff K fg \ (8X f 

6(9)J \Kgf Kgg) \SXg 



(16) 



In references on irreversible thermodynamics (e.g., de Groot and Mazur, 1962) the 
quantities on the left-hand side of (16) are called fluxes, and the (— 5X)s are thought of 
as the forces that drive the system back to equilibrium. We can thus think of the As 
as potentials that produce such forces. Linear homogeneous relations such as (16) were 
presumed by Onsager (1931), but here they arise quite naturally, and in (15) we observe 
the celebrated Onsager reciprocity relations. 

Suppose now that we add another constraint to the maximum-entropy construction 
by letting / be coupled to a weak thermal source. In addition, we shall specify that g 
is explicitly not driven, so that any internal changes in it can only be inferred from the 
changes in /. We thus set 5X g = in (16) and those equations reduce to 

Hf) = -K ff dX f = SQ f , 
8(g) = -K gf 8X f . 
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So for small variations the change in the coupled variable is essentially the source strength 
itself; the internal change in g is also proportional to that source strength, but modulated 
by the extent to which g and / are correlated: 

S(g) = ^ l SQ f , (18) 
K ff 

exhibiting what is sometimes referred to as mode-mode coupling. These expressions are 
precisely what one expects from a re-maximization of the entropy subject to a small change 
S(f). For example, if 5Qf > and / and g are positively correlated, K g f > 0, then we 
expect increases in the expectation values of both quantities, as well as a corresponding 
increase in the maximum entropy. 

Although this discussion of small departures from equilibrium is only a first step, it 
reinforces, and serves as a guide to, the important role of sources in any deeper theory. 
It also exhibits the structure of the first approximation, or linearization of such a the- 
ory, which is often a necessary consideration. We return to the essential aspects of that 
approximation a bit later. 

3. Sources and Thermal Driving 

We seek a description of macroscopic nonequilibrium behavior that is generated by 
an arbitrary source whose precise details may be unknown. One should be able to infer 
the presence of such a source from the data, and both the strength and rate of driving of 
that source should be all that are required for predicting reproducible effects. Given data 
- expectation values, say — that vary continuously in time, we infer a source at work 
and expect p to be a definite function of time, possibly evolving principally by external 
means. In I we argued that, because all probabilities are conditional on some kind of 
given information or hypothesis, P(Ai\I) can change in time only if the information / is 
changing in time, while the propositions {Ai} are taken as fixed. This then served as the 
basis for an abstract model of time-dependent probabilities. With this insight we can see 
how the Gibbs algorithm might be extended to time-varying macroscopic systems in a 
straightforward manner. 

As in I, information gathered in one time interval can certainly be followed by col- 
lection in another a short time later, and can continue to be collected in a series of such 
intervals, the entropy being re-maximized subject to all previous data after each interval. 
Now let those intervals become shorter and the intervals between them closer together, so 
that by an obvious limiting procedure they all blend into one continuous interval whose 
upper endpoint is always the current moment. Thus, there is nothing to prevent us from 
imagining a situation in which our information or data are continually changing in time. A 
rationale for envisioning re- maximization to occur at every moment, rather than all at once, 
can be found by again appealing to Boltzmann's expression for the entropy: Sb = InW. 
At any moment W is a measure of the phase volume of all those microstates compatible 
with the macroscopic constraints — and In W is the maximum of the information entropy 
at that instant. As Boltzmann realized, this is a valid representation of the maximized 
entropy even for a nonstationary state. It is essential to understand that W is a number 
representing the multiplicity of a macrostate that changes only as a result of changing 
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external constraints. It is not a descriptor of which microscopic arrangements are being 
realized by the system at the moment — there is no way we can ascertain that — but 
only a measure of how many such states may be compatible with the macrostate defined 
by those constraints. In principle we could always compute a W for a set of values of the 
macroscopic constraints without ever carrying out an experiment. Thus, we begin to see 
how an evolving entropy can possibly be related to the time-dependent process. 

There may seem to be a problem here for someone who thinks of probabilities as 
real physical entities, since it might be argued that the system cannot possibly respond 
fast enough for W to readjust its content instantaneously. But it is not the response of 
the system that is at issue here; only the set of possible microstates compatible with the 
present macroscopic constraints readjusts. Those potentialities always exist and need no 
physical signal to be realized. A retardation problem might exist if we were trying to follow 
the system's changing occupation of microstates, but we are not, because we cannot. The 
multiplicity W does not change just because the microstate occupied by the system changes; 
in equilibrium those changes go on continuously, but W remains essentially constant. Only 
variations in the macroscopic constraints can change W, and those are instantaneous and 
lead to immediate change in the maximum information entropy Sb- 

To introduce the notion of a general source let us consider a generic system described 
by a density matrix 

p4^ B+7C , (19) 

and a process that drives the variable B such that an amount AB is transferred into the 
system. That is, B is driven by some means other than dynamically, with no obvious 
effective Hamiltonian. In addition, the variable A is explicitly not driven, but can change 
only as a result of changes in B if A and B are correlated. Since there is no new information 
regarding A, even though it is free to readjust when B is changed, the Lagrange multiplier 
a must remain unchanged. We also add the further constraint on the process that C is to 
remain unchanged under transfer of AB. This is a generalization of the scenario described 
by (16), and can be summarized as follows: 

5a = 0, (A)^{A)', 

5(3 ^0, (B) -> (B) + AB , (2Q) 

51 = -^5(3, (C)^(C). 
J^cc 

This is the most general form of a constrained driven process, except for inclusion of a 
number of variables of each kind. Any such driving not tied to a specific dynamic term in 
a Hamiltonian will be referred to as thermal driving. A variable, and therefore the system 
itself, is said to be thermally driven if no new variables other than those constrained ex- 
perimentally are needed to characterize the resulting state, and if the Lagrange multipliers 
corresponding to variables other than those specified remain constant. As discussed in I, a 
major difference with purely dynamic driving is that the thermally-driven density matrix 
is not constrained to evolve by unitary transformation alone. 

Let us suppose that the system is in thermal equilibrium with time-independent Hamil- 
tonian in the past, and then at t = a source is turned on smoothly and specified to run 
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continuously, as described by its effect on the expectation value (F(t)). That is, F(t) is 
given throughout the changing interval [0, t] and is specified to continue to change in a 
known way until further noticed Although any complete theory of nonequilibrium must 
be a continuum field theory, we shall omit spatial dependence explicitly here in the interest 
of clarity and return to address that point later. For convenience we consider only a sin- 
gle driven operator; multiple operators, both driven and constrained, are readily included. 
Based on the probability model of I, the PME then provides the density matrix for thermal 
driving: 

i r r* i 

-(3H- / \{t')F{t')dt' 
Jo 



Pt = ^r exp 



-pH- [ \{t')F{t')dt' 
Jo 



(21) 



Z t [(3,\(t)} = Tr exp 
and the Lagrange-multiplier function is formally obtained from 

{F ^'--sk lnZt - (22) 

for t in the driving interval. Reference to the equilibrium state is made explicit not only 
because it provides a measure of how far the system is removed from equilibrium, but also 
because it removes all uncertainty as to the previous history of the system prior to intro- 
duction of the external source; clearly, these are not essential features of the construction. 

Since pt can now be considered an explicit function of t, we can employ the operator 
identity d x e A ^ = e A ^d x A to compute the time derivative: 



d tPt = Pt X(t)[(F(t)} t -F(t)}, (23) 
where the overline denotes a generalized Kubo transform with respect to the operator lnpt: 



F(t) = [ e~ ulnpt F(t)e ulnpt du , 
Jo 



(24) 



which arises here from the possible noncommutativity of F(t) with itself at different times. 

The expression (23) has the form of what is often called a 'master equation', but it 
has an entirely different origin and is exact; it is, in fact, the dtp term in the equation 
of motion (1-40). Because X(t) is defined only on the information-gathering interval [0, £], 
Eq.(23) just specifies the rate at which p t is changing in that interval. Although p t does 
not evolve under time-independent H in the Heisenberg picture, in this case it does evolve 
explicitly, and in the Schrodinger picture this time variation will be in addition to the 
canonical time evolution. In turn, an unambiguous time dependence for the entropy is 
implied, as follows. 

The theoretical maximum entropy St = — kTr[p t lnp^] is obtained explicitly by substi- 
tution from (21), 

^St = lnZ t + (3(H) t + X(t'){F(t%dt'; (25) 



t The lower limit of the driving interval is chosen as only for convenience. 
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it is the continuously re-maximized information entropy. Equation (25) indicates explicitly 
that (H ) t changes only as a result of changes in, and correlation with F. The constraint 
that H is explicitly not driven implies that {H) t and (F(t'))t are no longer independent, 
and that means that X(t) cannot be determined directly from St by functional differentia- 
tion in (25); this has important consequences. 

The expectation value of another operator at time t is (C) t = Tr[p t C], and direct 
differentiation yields 

j t (C(t))t = Tr[C(t)d t Pt + PtC(t)] 

= (C(t)) t -X(t)K t CF (t,t), 

where the superposed dot denotes a total time derivative. We have here introduced the 

covariance function 

6(C(t)) t 



K* CF {t\ t) ee (F(t')C(t)) t - (F(t')) t (C(t)) t = ~-^f , (27) 

which is a quantum mechanical generalization of the covariance (15). Note that all of the 
preceding entities are completely nonlinear, in that expectation values, Kubo transforms, 
and covariance functions are all written in terms of the density matrix p u which is the 
meaning of the superscript t on K l CF . Although time-translation invariance is not a prop- 
erty of the general nonequilibrium system, it is not difficult to show that the reciprocity 
relation K^ F (f,t) = K FC (t,t') is valid. 

Let us introduce a new notation into (26), which at first appears to be only a conve- 
nience: 

°c(t) = J f (C(t))t ~ (C(t))t = -Kmhrit, t) . (28) 

For a number of choices of C and F the equal-time covariance function vanishes, but if 
C = F an illuminating interpretation first noticed by Mitchell (1967) emerges: 

M«)-£dW>. -c*M>, (29) 

= -\(t)K' FF (t,t) . 

Owing to the specification of thermal driving, d(F(t))t/dt is the total time rate-of-change 
of (F(t))t in the system at time t, whereas (F(t))t is the rate of change produced by 
internal relaxation. Hence, o p(t) must be the rate at which F is driven or transferred by 
the external source, and is often what is measured or controlled experimentally. One need 
know nothing else about the details of the source, because its total effect on the system 
is expressed by the second equality in (29). If the source strength is given, then (29) is a 
nonlinear transcendental equation determining the Lagrange multiplier function X(t). 

An important reason for eventually including spatial dependence is that we can now 
derive the macroscopic equations of motion. For example, if F(t) is one of the conserved 
densities e(x, t) in a simple fluid and J(x, t) the corresponding current density, then the 
local microscopic continuity equation 

e(x,£) + V • J(x,t) = (30) 
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is satisfied irrespective of the the state of the system. When this is substituted into (29) 
we obtain the macroscopic conservation law 

^<e(x, t)) t + V • (J(x, t)) t = <r e ( x , t) , (31) 

which is completely nonlinear. Specification of sources therefore provides automatically the 
thermokinetic equations of motion; for example, if e is the momentum density mj(x, t), 
so that J is the stress tensor T^, then a series of transformations turns (31) into the 
Navier-Stokes equations of fluid dynamics. 

NONEQUILIBRIUM THERMODYNAMICS 

The notion of thermal driving provides a basis for nonequilibrium thermodynamics, 
which can be developed in much the same way as is done for the equilibrium theory (e.g., 
Grandy, 1987). As with that case, the operator F can also depend on an external variable 
a, so that at time t the entropy is St = St[(H) t , (F(t))f, a}; of course, we could also include 
a number of other measured variables {i^}, though only H and F will be employed here. 
But now St is also a function of time and, from (25), its total time derivative is 

^§K^) 4 + ^- A(t) /' A(f ' )A - (M ' )rf< ' (32) 

Although d t Z t contributes to S t , its contribution is cancelled because 

d t \nZ t = -\(t)(F(t))t, (33) 

which also provides a novel representation for Zt upon integration. In principle, then, one 
can follow the increase (or decrease) of entropy in the presence of external sources (or 
sinks). 

The most common type of external variable a is the system volume V, so that in 
the equilibrium theory (d(H) / 'dV)dV = —PdV is an element of work. This suggests a 
general interpretation of the first term on the right-hand side of (32). As an example, in 
the present scenario consider the simple process of an adiabatic free expansion of a gas, 
wherein only the work term is involved in (32). We can now model this by specifying a 
form for a = V; for example, V(t) = Vq{2 — e~ bt ) would, for b very large, rapidly inflate 
the volume to double its size over an interval from t = to some later time r. One also 
needs an equation of state for the gas, but usually the pressure is proportional to V~ x and 
therefore decreases exponentially as well. In the case of an ideal gas, integration of this 
form for St over (0, r) yields the expected change S T — S = kN In 2. This result is almost 
independent of the model as long as V(r) ~ 2Vq. 

Ordinarily a = 0. In this case we can also explicitly evaluate the term containing the 
Hamiltonian and rewrite (31) as 

1 dS f l 

-^ = -p\(t)K t HF (t,o)-\(t) y o mK^t^dt* 

= n/ F (t)<T F (t) , 
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where we have employed (29) and defined a new parameter 



^4tfH V) ftt¥*'- (35) 

Although this expression for 7 at first glance seems only a bookkeeping convenience, it 
is actually of some physical significance, as suggested by (20). As noted above, the thermal 
driving constraint on H prevents {H) t and (F(t)) t from being completely independent; 
indeed, neither of them is independent of (F(t')) t - In turn, and unlike the equilibrium 
case, d{f rn )/dX n and d\ n /d(f m ) are no longer the respective elements of a pair of mutually 
inverse matrices. Thus, 8S t /S{F(t)) t does not determine A(t); rather, from (25), 

S3, _6(H), S M t) *A(Q df , (36) 



S(F(t)) t SX(t) 5{F(t)) t J SX(t) 5{F(t)} 

Owing to interdependencies we can now write SX(t)/S(F(t))t = l/Kp F (t,t), and hence 
the right-hand side of (36) is just 7f(£), which now has the general definition 

F ^ ^ ^5{F(t))t ^ thermal ^ ^ 



driving 



The subscript "thermal driving" reminds us that this derivative is somewhat different than 
its usual form in equilibrium. When the source strength o~ p(t) is specified the Lagrange 
multiplier itself is determined from (29) . 

Physically, 7f is a transfer potential in the same sense that the As in Eq.(16) are 
thought of as potentials. Just as products of potentials and expectation values appear in 
the structure of the equilibrium entropy, in thermal driving the entropy production (34) 
is always a sum of products of transfer potentials and source terms measuring the rate 
of transfer. So, the entropy production is not in general given by products of 'fluxes' 
and 'forces', and St and St are not simple generalizations of equilibrium quantities. But 
the ordinary potentials also play another role in equilibrium: if two systems in contact 
can exchange energy and particles, then they are in equilibrium if the temperatures and 
chemical potentials of the two are equal. Similarly, if two systems can exchange quantities 
Fi under thermal driving, then the conditions for migrational equilibrium at time t are 

7^(^)1=7^(^)2. (38) 

Migrational equilibrium in stationary processes is discussed, for example, by Tykodi (1967). 

What is the physical interpretation to be given to St? Clearly it refers only to the 
information encoded in the distribution of (21) and cannot refer to the internal entropy 
of the system. In equilibrium the maximum of this information entropy is the same as 
the experimental entropy, but that is not necessarily the case here. For example, if the 
driving is removed at time t = t±, then St ± in (25) can only provide the entropy of that 
nonequilibrium state; its value will remain the same during subsequent relaxation, owing 
to unitary time evolution. Although the maximum information (or theoretical) entropy 
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provides a complete description of the system based on all known physical constraints on 
that system, it cannot describe the ensuing relaxation, for it contains no new information 
about that process. Nevertheless, St does have a definite physical interpretation. 

The form of o F in (29) suggests a natural separation of the entropy if that expression 
is substituted into the second line of (34) : 

St = lF(t)(j t (F(t))t-{F(t))?J . (39) 

Thus, St has the qualitative form Q/T, as intuition might have suggested. The first term 
on the right-hand side must represent the total time rate-of-change of entropy Stot arising 
from the thermal driving of F(t), whereas the second term is the rate-of-change of internal 
entropy Si nt owing to relaxation. Thus, the total entropy production can be written 

S t ot(t) = St + S int (t), (40) 

where the entropy production of transfer owing to the external source, St, is given by (34). 
This latter quantity is a function only of the driven variable F(t), whereas the internal 
entropy depends on all variables, driven or not, necessary to describe the nonequilibrium 
state and is determined by the various relaxation processes taking place in the system. 
Calculation of Si nt , of course, depends on a rather detailed model of the system; we'll have 
more to say on this below. t 

In an equilibrium system the major role of S is associated with the Second Law, and 
this law in its traditional form has little to say about nonequilibrium processes. In these 
latter processes, however, it is St, rather than St itself that plays the major role, as is seen 
in (34)-(37). That is, St governs the transfer process in terms of the rate of driving and the 
transfer potential, in much the same way that S governs the direction of changes between 
equilibrium states through dQ/T. In nonequilibrium processes St also governs the rate; 
this is true even in the steady state when one takes into account sources and sinks. 

The distinction between theoretical entropy in equilibrium scenarios and in nonequi- 
librium processes cannot be emphasized enough. If external forces are removed, it is a 
mathematical theorem that neither p t nor St can evolve into their equilibrium counter- 
parts. This is a singular limit, as discussed earlier, and unless these distinctions are clearly 
recognized few real advances can be made in nonequilibrium statistical mechanics. 

Constant Driving Rate and Spatial Variation 

To complete the general development, logical consistency requires an examination of 
thermal driving at a constant rate. For this purpose it will first be useful to record the 
generalizations of the primary equations of thermal driving to include spatial coordinates: 

d tPt =p t I A(x', t) [<F(x', t)) t - FW)\ d 3 x' , (41) 



' Equation (40) is reminiscent of similar expressions for entropy changes, such as dS = dS ex t + 
dSi n t, that can be found in works on phenomenological nonequilibrium thermodynamics (e.g., de 
Groot and Mazur, 1962). 
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^S t = In Z t + (3(H) t + J d 3 x' J dt'\(x.',t'){F(x.',t')) t , (42) 
\s t = -P I A(x',£)i^ F (x',£)dV 

r r r* (43) 

- J d 3 x"\(x",t) J d 3 x' J dt , X(x! , ,l/)K t FF (x.",t;x',lf), 

cx F (x, t) = - J A(x', t)K* FF (x!, t; x, t) . (44) 
This last expression can be inverted by introducing an inverse integral operator: 

A(x, t) = - J [4 F (x', t; x, £)] ~V F (x', t) d 3 x' , (45) 

which is a nonlinear integral equation for A(x, t). Thus, the right-hand side of (45) is really 
only a shorthand notation for the iterated solution. Upon substitution of (45) into (43) 
we find that 

^S t = J l F (^t)a F (^t)d 3 x, (46) 



(47) 



where 

7f(M) = /3 J dV[iir|, F .(x,t;x / ,t)]" 1 iiC| rF (x',t) 

y rfVy dV'jT dt'X(x",t)[K t FF (x,t;x',t)]~ 1 K t FF (x',t;x",t'). 

We can verify this expression for 7 F from the more general definition 

7 F (x,t)^( ^ ) , (48) 

(X, tj)t / thermal 
driving 

if we note two properties of functional differentiation. First, the ordinary chain rule for 
partial differentiation of F[x(s) : y(s)] with respect to s, 

OF dF dx OF dy 
ds dx ds dy ds ' 

generalizes to 



*(<3(M)) _ [ 6(G(x,t))6(\ F (x',t) 3 , 
<5(F(x,£)) 7 <5A F (x',£) <5(F(x,£)) * 

= J K t GF {y^ \t;x,t) [K FF (^t-^ : t)]~ l d 3 x' . 
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(49) 



for example. Second, in Eq.(42) for St the upper limit t on the time integral, and the 
subscript on (H) t , prevent the functional derivative from yielding merely A(x, t), which is 
determined by <tf(x, t) at any rate. Rather, we obtain (47) for 7_f( x i^)- 

Specification of constant driving means that of is constant in time, and from (29) 
or (44) this in turn implies that A(x, t) must actually be independent of time. This last 
assertion follows because the covariance function in these equations is time independent, 
owing to the re-emergence of unitary time evolution in the absence of internal time vari- 
ation. That is, the integrals in (21), generalized to include spatial variables, can now be 
rewritten in the form 



But now the form of the time integral no longer makes sense in the context of time- 
independent driving. 

If a constant rate of driving is specified as a constraint on the initial probability 
distribution we take this to mean that the initial data were constant in the distant past, 
and at least up to the time of observation. In requiring this one faces the possibility of 
a divergent integral, so that it is necessary to regularize the integral, along the lines of 
methods often employed in quantum field theory. In the present case we rewrite the time 
integral in (50) as a time average over the past: 



This, however, is just the diagonal part of the operator F(x!) as given by Eq.(ll), and 
hence constant driving corresponds with our definition of the steady state. In this scenario 
we can then replace all the time integrations over operators by the diagonal parts of those 
operators and omit all time dependence. We see that, in the sense of this procedure, the 
steady state is also a singular limit of the general nonequilibrium state, in that the latter 
does not reduce in a completely straightforward mathematical way to the former. 

In the steady state we expect time derivatives of all expectation values to vanish; 
hence from (29) we have the further implication that the constant rate of driving is exactly 
balanced by the rate of internal relaxation. This is how the system responds to steady 
currents. 

Although there exist stationary currents within the system, the steady driving takes 
place in the terminal parts, or boundaries of the system, and such currents imply irre- 
versible dissipation. There must then be an overall rate of dissipation or entropy produc- 
tion generated by the external sources. This rate is provided by Eq.(46), now rewritten in 
the form 




(50) 




(51) 




(52) 



The general definition of 7f(x) still applies, but the explicit form is now 




(53) 
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4. The Linear Approximation 

Much, though not all, of the work on macroscopic nonequilibrium phenomena has of 
necessity centered on small departures from equilibrium, or the linear approximation, so 
that it is of some value to outline that reduction of the present theory and discuss briefly 
some applications. We envision situations in which the system has been in thermal equi- 
librium in the remote past and later found to produce data of the form considered above. 
By considering both pieces of data we obtain a measure of the departure from equilibrium. 
In describing the general method of linearization the character of the perturbing term and 
the scenario under consideration are immaterial; hence, we can take the distribution (21) 
with integration limits replaced by the space-time region R as our generic model and, for 
brevity, temporarily omit space dependences. ^ Thus, we consider the model 

P = | exp - J X(t)F(t) dt} , (54) 

Z[P, X(t)] = Trexp J X(t)F(t) dtj , (55) 

where (3 refers to the temperature of the previous equilibrium state — no other value of (3 
makes sense until the system returns to equilibrium. 

By linear approximation we mean "linear in the departure from equilibrium." In the 
present case that means that the entire integral in (54) and (55) is in some sense small. 
An expansion of the exponential operator follows from repeated application of the identity 



e A+B = e A 



f 

JO 



1 + / e~ xA Be x{A+B) dx 



(56) 



where B is the small perturbation. The first-order, or linear approximation to the ex- 
pectation value of another operator C is (Heims and Jaynes, 1962; Jaynes, 1979; Grandy, 
1988) 

(C) ~ (C% - / (e~ xA Be xA C) dx + (B) (C) Q , (57) 
Jo 

where (B) = Ti(e A B). In (57) we again encounter the Kubo transform of the operator 
B with respect to A, the nonlinear form of which was introduced in (24). 

Application of this approximation scheme to (54) and (55) reveals that the leading- 
order departure of the expectation value of C at time t from its equilibrium value is 

(C(t)) - (C) = - / K CF (t, t')X(t') dt' , (58) 

JR 

where Kcf = Kcf ^ s ^ ne linearized version of the covariance function defined in (27): 

K CF {t,t') = (W)C(t)) - (F)o(C) 

S(C(t)) (59) 
SX(t) ' 



t The equilibrium distribution is taken as canonical only for convenience; for example, one could 
just as well use the grand canonical form, as well as include different types of particle. 
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and (• ■ -)o is an expectation value in terms of the equilibrium distribution p . Time inde- 
pendence of the Hamiltonian confers the same property upon the single-operator expec- 
tations, and also guarantees time-translation invariance: KcF(t,t') = KcF(t — t'). One 
verifies the reciprocity relation 



from a change of variables and cyclic invariance of the trace. Note that it is always the 
second variable that carries the Kubo transform. If C and F are Hermitian, Kqf is real 
and Kff > 0. In this case Kcf has all the properties of a scalar product on a linear vector 
space, and thus satisfies the Schwarz inequality: KccKff — Kcf — 5 with equality if 
and only if C = cF, with c a real constant. 

The covariance function (59) clearly depends only on equilibrium properties of the 
system. Quite generally, then, small departures from equilibrium caused by anything are 
described principally by equilibrium fluctuations. While this provides some useful physical 
insight, the other side of the coin is that covariance functions are exceedingly difficult to 
calculate for interacting particles, other than in some kind of perturbation theory. The lin- 
ear approximation represents considerable progress, but formidable mathematical barriers 
remain. In practice, however, it is usually the relations among these and other quantities 
that interest us; after all, we seldom evaluate from first principles the derivatives in the 
Maxwell relations, yet they provide us with important insights. Linear hydrodynamics 
provides one area in which various approximation schemes for correlation functions have 
proved fruitful. 

While the Lagrange multiplier function X(t) is determined formally by (7), one suspects 
that if we set C = F and restrict t to the region R, then (58) becomes a Fredholm integral 
equation determining X(t) in the only interval in which it is defined. This indeed turns out 
to be the case, though the demonstration that the two procedures are equivalent requires a 
little effort (Grandy, 1988). This is, in fact, a very rich result, and to discuss it in slightly 
more detail it will be convenient to specify R more definitely, as [— r, 0], say. Thus, the 
expression 



is now seen to have several interpretations as t ranges over (—00,00). When t > it 
gives the predicted future of F(t); with — r < t < it provides a linear integral equation 
determining X(t); and when t < — r it yields the retrodicted past of F(t). This last 
observation underscores the facts that KpF{t) is not necessarily a causal function unless 
required to be so, and that these expressions are based on probable inference; in physical 
applications the dynamics enters into computation of the covariance function, but does 
not dictate its interpretation in various time domains. Although physical influences must 
propagate forward in time, logical inferences about the present can affect our knowledge 
of the past as well as the future. Retrodiction, of course, is at the heart of fields such as 
aarcheology cosmology, geology, and paleontology. 

When the perturbed system is spatially nonuniform we find that (58) and (59) are 



K CF (t-t') =K FC (t'-t) 



(60) 




(61) 
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replaced by 

(C(x, t)) - (C(x)) = - / Kcf(x, t; x', t')\(x', t') dV dt' , (62) 

JR 

K C f(x, t; x', t') = <F(x',t')C(x, *))o - (F(x / )) (C(x)) , (63) 

so that in its causal domain if^^x, t; x', £') takes the form of a Green function. Note 
that the single-operator expectation values are also independent of x in an initially homo- 
geneous system, and that the generalization to include a number of operators Fk(x.,t) is 
straightforward. 

If the equilibrium system is also space-translation invariant it is useful to employ the 
notation r = x — x'. Generally, the operators encountered in covariance functions possess 
definite transformation properties under space inversion (parity) and time reversal. Under 
the former A(r, r) becomes PaA(—t, t), Pa = ±1, and under the latter TaA(t, —t), Ta = 
±1. For operators describing a simple fluid, say , PT = +1 and one verifies that the full 
reciprocity relation holds: 

K cf {y,t) = K fc {v,t). (64) 

The efficacy of these equations of the linear approximation will become apparent as we 
present some sample applications. 



Linear Transport Processes 

The generic model for a macroscopic fluid is most readily described as a continuum in 
terms of various densities, and representations in terms of quantum-mechanical operators 
are defined in terms of field operators in a Fock representation {e.g., Fetter and Walecka, 
1971). The three basic density operators in the fluid are the number density n, momentum 
density mj, and energy density h, where j is the particle current- density operator. Unless 
so specified, these generally have no explicit time dependence, so that their equations of 
motion in the Heisenberg picture are 

n(x, t) = n(x, £)] , (65a) 

™j(x, t) = -[H, mj(x, t)] , (65b) 

h(x,t) = ^[H,h(x,t)} . (65c) 

But the left-hand sides of these equations are also involved in statements of the local 
microscopic conservation laws in the continuum, which usually relate time derivatives of 
densities to divergences of the corresponding currents. The differential conservation laws 
are thus obtained by evaluating the commutators on the right-hand sides in the forms 

(66a) 
(66b) 
(66c) 



n(x, t) = 


-V 


• j(M) , 


mj(x,t) = 


-V 


•T(x,t), 


/i(x,£) = 


-V 


• q(x, t) . 
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The superposed dot in these equations indicates a total time derivative. In the absence 
of external forces and sources (65) are equivalent to unitary transformations, and the 
Hamiltonian and total-number operator, respectively, are given by 

H = Jh(x,t)d 3 x, N = Jn(x,t)d 3 x, (67) 

both independent of time. 

The current density j is just the usual quantum- mechanical probability current den- 
sity, so that (66a) is easily verified. Identification of the energy current density q and 
stress tensor T, however, is far from straightforward; in fact, they may not be uniquely 
defined for arbitrary particle-particle interactions. But if the Hamiltonian is rotationally 
invariant we can restrict the discussion to spherically-symmetric two-body potentials. Two 
further symmetry properties arise from time independence and spatial uniformity in the 
equilibrium system: time-translation and space-translation invariance, respectively. These 
latter two invariances are expressed in terms of volume-integrated, or total energy, num- 
ber, and momentum operators, so that the commutators [H, P], [H,N], [P,N] all vanish. 
Specification of these symmetry properties defines a simple fluid, and the operators q and 
T can be identified uniquely by evaluation of the commutators in Eqs.(65b,c). The algebra 
is tedious and the results are given, for example, by Puff and Gillis (1968), and Grandy 
(1988). Thus, the five local microscopic conservation laws (66) completely characterize the 
simple fluid and lead to five long-lived hydrodynamic modes. Local disturbances of these 
quantities cannot be dissipated locally, but must spread out over the entire system. 

As a first application of the linear theory we return to the steady-state scenario of 
Eqs.(12) and (13) and also incorporate a term —f3H in the exponentials to character- 
ize an earlier equilibrium reference state. Denoting the deviation from equilibrium as 
AF(x) = .F(x) — (F(x)) , we find that in linear approximation another operator C will 
have expectation value 



(AC(x)) ss = - / A(x')Kcf(x - x') d 3 x' 

JR 



+ lim I d s x' [ e et \(x.')K c p(x-x.',t)dt, 

e ^ 0+ JR J-oo 



(68) 



where we have employed the expression (11) for the diagonal part of an operator, and 
subscripts ss refer to the steady-state distribution. Specify -F(x) to be one of the fluid 
densities <i(x), so that the continuity equations (66) lead to the identity 

j t K dB (x,t) = -V-K iB (x,t), (69) 

and thus K c ^ in (68) can be replaced by —V' • Kcj- Let -R(x) be the system volume V, 
and presume Kcj to vanish at large distances. An integration by parts then reduces (68) 
to 

(AC(x)) ss = - / A(x')Kcd(x - x') d 3 x' 

JV , f o (70) 

+ lim / d 3 x' / e et V'A(x') ■ K CJ U - x\t) dt , 

e^0+J v 
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in which we have dropped the surface term. 

Classical hydrodynamics corresponds to a long-wavelength approximation by presum- 
ing that VA varies so slowly that it is effectively constant over the range for which Kcj 
is appreciable.^ With this in mind we can extract the gradient from the integral and write 



which is the fundamental equation describing linear transport processes in the steady state. 
The integration region v is the correlation volume, ooutsideof which the correlations vanish; 
it is introduced here simply as a reminder that the spatial correlations are presumed to be 
of short range. 

As an example, let d be the number density n with gradient characterized by the 
deviation An(x) = n(x) — (n) . The specified density gradient and the predicted current 
density, respectively, are then 



where the limit e — > + is understood. We have noted that the second term of the first 
line in (72) and the first term of the first line in (73) vanish by symmetry. 

Now take the gradient in (72), make the long- wavelength approximation, and eliminate 
VA between this result and (73), which leads to the relation 




(71) 




(72) 




(73) 



U(x)> 



f v K nn (x-x.')d?x' 
D(x)-V(n(x)>, 



•V(n(x)> 



(74) 



We presume that the fluctuations are not correlated over the entire volume. 
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with the proviso that e — > + . This is Fick's law of diffusion, in which we have identified 
the diffusion tensor D that can now be calculated in principle from microscopic dynamics; 
owing to spatial uniformity in the equilibrium system -D(x) is actually independent of x. 
For more general nonequilibrium states the same type of calculation produces a quantity 
D(x, t) having the same form as that in (74), and the long- wavelength approximation also 
involves one of short memory. (By 'short memory' we mean that recent information is the 
most relevant, not that the system somehow forgets.) 

It is remarkable that linear constitutive equations such as Fick's law arise from almost 
nothing more than having some kind of data available over a space-time region. These 
relations have long been characterized as phenomeno logical, since they are not derived 
from dynamical laws. We now see why this is so, for the derivation here shows that they 
are actually laws of of inference. Indeed, what we usually mean by 'phenomenologicaP 
is 'inferred from experience', a notion here put on a sound footing through probability 
theory. When they are coupled with the corresponding conservation laws, however, one 
does obtain macroscopic dynamical laws, such as the diffusion equation. 

Because it involves a slightly different procedure, and will provide a further example 
below, let us consider thermal conductivity (which need not be restricted to fluids). A 
steady gradient in energy density is specified in the form of a deviation A/i(x) = h(x.) — (h)o. 
By a calculation similar to the above we find for the expected steady-state heat current 



where the limit e — > + is understood, and we have not yet invoked the long- wavelength 
limit. In this case we do not eliminate VA, for it contains the gradient of interest. Both 
dimensionally, and as dictated by the physical scenario, A must be /3(x) = [/cT(x)] _1 , 
a space-dependent temperature function. Although such a quantity may be difficult to 
measure in general, it is well-defined in the steady state. With this substitution the long- 
wavelength approximation of constant temperature gradient in (75) yields 



in which we identify the thermal conductivity tensor k, which again is independent of x. 
This is Fourier's law of thermal conductivity; it applies to solids as well as fluids, but 
calculation of the covariance function remains a challenge. It is left to the reader to verify 
that k, as well as D in (74), are positive. 

A common model employing (76) is that of a uniform conducting rod of length L and 
thermal conductivity k. We can calculate the constant rate of transfer of entropy from 
the source to the sink by means of (52), in which the transfer potential 7(x) is simply the 
spatial temperature distribtution /?(x), and cr(x) is the (constant) rate of driving on the 
end boundaries of the rod. In this case the driving rate is given by the heat current (q) 




(75) 




(76) 



k ■ VT(x) , 
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itself, inserting thermal energy at one end and taking it out at the other. Hence, 

\s t = £ ^(-«VT) [5^,0 - 6 XtL ] dx 

k (Th - Tc) 2 
~ L T H T C ' 

which is identical to the more intuitively obtained result (e.g., Palffy-Muhoray, 2001). 
Although (52) itself is completely nonlinear, one notes that we have employed the linear 
form of Fourier's law (76) for the current. This calculation illustrates the importance of 
boundary conditions in describing stationary processes; Tykodi (1967) has also emphasized 
the role of terminal parts in describing the steady state. 

Linear Response Theory 

An important feature of the thermal driving mechanism is that the actual details of 
the thermal driving source are irrelevant, and only the rates and strengths at which system 
variables are driven enter the equations. It should make no difference in many situations 
whether the driving is thermal or mechanical; we examine the latter context here. 

The theory of dynamical response was described very briefly in Eqs.(I-5)-(I-8), and 
the linear version follows as described there. The underlying scenario is that a well-defined 
external field is imposed on a system that has been in thermal equilibrium in the remote 
past, as described by the Hamiltonian Hq. It is then presumed that the response to this 
disturbance can be derived by adding a time-dependent term to the Hamiltonian, so that 
effectively H = Hq — Fv(t), t > 0, where v(t) describes the external field and F is a system 
operator to which it couples. Some of the difficulties with this approach were sketched 
in I, including the observation that pit) can only evolve unitarily. We now see that these 
problems can be resolved by noting that dynamical response is just a special case of thermal 
driving. 

For eventual comparison with the results of linear response theory we shall need 
an identity for the time derivative of the covariance function. Direct calculation in the 
definition (59) yields 

± KcF(T) = ±. (lC ,F(r) ] ) m 
= -/TVcfM, 

where <Pcf is the linear response function. Clearly, the covariance function contains a good 
deal more information than does the dynamic response function. 

The derivation of the generic maximum-entropy distribution in (1-14) disguises a 
subtle point regarding that procedure. We note from (1-16) that the Lagrange multiplier 
A can also be determined from the maximum entropy: 

^\WY (79) 

Together with (1-15) this reveals a reciprocity implying that the probability distribution 
can be obtained by specifying either (f) or A. An example of this choice is illustrated in 
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the canonical distribution (1-10), which could be obtained by specifying either the energy 
or the temperature; this option was also exercised in the model of spatial inhomogeneity 
of Eq.(lO). Thus, we return to Eqs.(21), replacing H with Hq, and let X(t') be the in- 
dependent variable. In linear approximation (29) expresses X(t) directly in terms of the 
source strength, or driving rate, and dimensional considerations suggest that we write this 
variable in the form 



d 



x(t') = p-[e(t-t f )v(t')} 



d 



-s(t-t')v(t') + e(t-t')—v(t') 



(80) 



with the condition that v(0) = 0. The step-function 9{t — t') is included in (80) because A 
is defined only on the interval [0, t]. 

Substitution of (80) into (21) yields the distribution relevant to a well-defined external 

field, 



Pt 



1 

Z~t 
1 



exp 



-PHo + P [ Idit-t^-eit-t^^vit^Fit^dt' 
J L at J 



exp 



-f3H + [3 I v{t')F{t')dt' 
o 



(81) 



and Z t , as usual, is the trace of the numerator. Although the exponential contains what 
appears to be an effective Hamiltonian, we do not assert that f Q v(t')F(t') dt' is an addition 
to the equilibrium Hamiltonian Hq; there is no rationale of any kind for such an assertion. 
The Lagrange multiplier function A(t) is a macroscopic quantity, as is its expression as an 
independent variable in (80). The linear approximation (58), along with the identity ((78), 
yields the departure from equilibrium of the expected value of another operator C at any 
future time t under driving by the external field: 



(C(t)) - (C) = (3 f v{t')K cP {t - O dt' 
Jo 

= j\(t')^K CF {t-t')dt' 

= [ v{t')(j)CF{t-t')dt' , 

Jo 



(82) 



which is precisely the result obtained in linear response theory. But now we also have 
the time-evolved probability distribution (81) from which we can develop the associated 
thermodynamics. Equation (82) confirms that, at least linearly, both pt and a unitarily 
evolved p(t) will predict the same expectation values. But, as suggested following (78), 
p(t) contains no more macroscopic information than it had to begin with. 

As an example of an external source producing a time-varying field, suppose a com- 
ponent of electric polarization Mi(t) is specified, leading to the density matrix 



1 



Pt 



exp 



-0H O + I \i(t')Mi(t' dt' 
'o 



(83) 



24 



We presume no spontaneous polarization, so that in linear approximation the expectation 
of another component at time t is 




Jo 



(84) 



Now, with the additional knowledge that (84) is the result of turning on an external 
field one might be led to think that the Lagrange multiplier is simply a field component, 
say Ei(t). But (80) shows that, even when the effect is to add a time-dependent term to the 
Hamiltonian, the actual source term is somewhat more complicated; only the 5-function 
term in (80) corresponds to that possibility, and the actual source term also describes the 
rate of change of the field. This again illustrates the earlier observation that the covariance 
function contains much more information than the dynamic response function. 

With (80) we can rewrite (84) explicitly as 



which is just the result obtained from the theory of dynamic response. But we've uncovered 
much more, because now one can do thermodynamics. In the present scenario we have 
specified thermal driving of the polarization and incorporated that into a density matrix; 
additionally, the Lagrange multiplier has been chosen to be the independent variable cor- 
responding to an external field, which allows us to identify the source strength. Thus, we 
have a definite expression for the time-dependent entropy of the ensuing nonequilibrium 
state: 



where the second line is the linear approximation and we have identified the entropy of 
the equilibrium system as S = k lnZ + k/3(H } . In the case of dynamic response, if one 
makes the linear approximation to pit) in (1-6) and computes the entropy similarly, it is 
found that S(t) — Sq vanishes identically, as expected. With (86), however, the entropy 
difference can also be written in terms of the linear response function: 



These remarks strongly suggest that the proper theory of response to a dynamical pertur- 
bation is to be found as a special case of thermal driving. 



(M 3 {t)) = (3(M,{t)M 3 {t)) Q E,{t) 




(85) 




(86) 




(87) 
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Relaxation 



When external sources are removed we expect the system to relax to a (possibly new) 
state of thermal equilibrium. If the driving ceases at time t = t±, say, then from that point 
on the system is described by (21) with the replacement t — > t\ everywhere, barring any 
further external influence. These equations define the nonequilibrium state at t = t±, from 
which the subsequent behavior can be predicted. 

As discussed earlier, S tl as given by (25) cannot evolve to the entropy of some equi- 
librium state, for the same reason that pt x cannot evolve to a canonical equilibrium dis- 
tribution; both evolve from t = t\ under unitary transformation. It should be sufficient, 
however, to show that the macrovariables describing the thermodynamic system, such as 
(F(t))t 17 may relax to a set of equilibrium values. Then, with those predicted values, we 
can construct a new canonical density matrix via entropy maximization that will describe 
the new equilibrium state. The value S tl remains the entropy of the nonequilibrium state 
at the time the driving was removed. 

Calculation of the exact expectation values is essentially intractable, of course, so we 
again employ the linear approximation. For example, at time t > t\ the expectation of 
F(t) itself is 



where we've utilized (29) . Although everything on the right-hand side of (88) is presumably 
known, we see that the actual details of the relaxation process depend crucially on the 
behavior of KpF(t — t') for t > t\. 

In the discussion following Eq.(60) we noted that the covariance functions satisfy the 
Schwarz inequality. From this we can see that the ratio r(t — t') = \KpF{t — t')/ Kff{^)\ 
in (88), and therefore the integrand, reach their maxima at t' = t where r(0) = 1. Further, 
r(t — t') is less than unity for t' < t, and again for all t > t\; the exact magnitude of r 
depends on the decay properties of Kppit — t'). In any event, the major contribution to 
the integral arises from the region around the cutoff t = t\. The relaxation time r can be 
estimated by studying the asymptotic properties of the time-derivative of AF(t), which 
in turn requires an examination of the time-derivative of Kppit — t'). From (78), we are 
thus seeking a time t 2 for which 



by some criterion. Then, r ~ £2 — t\. 

For many covariance functions the ratio r(t — t') in (88) will tend to some constant 
value as tjt\ becomes large, while others may tend to zero. For example, if we turn off 
the burner under a pot of water at t = t\, the total energy of the equilibrium system will 
be (i?(ti))t 1 , so that Kee would be expected to reach its nonzero asymptotic form very 
quickly. But in the polarization example of (84) we expect the correlations to decay to zero 
as the system relaxes back to the unpolarized state; this may, or may not, be rapid. One 




(88) 



|([F(0,F(t)]>o|«l, 



(89) 
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can only uncover the particular behavior from a detailed study of the covariance functions 
as determined by the relaxation mechanisms specific to the system, which are generally 
governed by particle interactions. 

A final point we can make here concerns the rate-of-change of internal entropy, Si n t- 
When the source is removed the entropy production of transfer immediately vanishes for 
t > t\. Equation (40) then implies that the total rate of entropy production is entirely 
that of relaxation, and the compelling conclusion is that Si n t is actually the relaxation rate 
and may be observable. This interpretation should also be valid when St 7^ 0, which is 
strongly indicated in the steady state scenario. 

5. Summary 

The aim of this discussion has been to expand the concept of theoretical entropy 
in equilibrium thermodynamics to encompass macroscopic systems evolving in time. In 
doing so we find that the maximum information entropy St, while providing a complete 
description of the nonequilibrium state at any instant, does not assume the dominant role 
it does in an equilibrium context. Rather, the rates and directions of processes are the 
most important features of nonequilibrium systems, and the rate of entropy production S t 
takes the form of a transfer potential times a rate of transfer, or a generalized intuitive 
form of Q/T. This suggests that in nonequilibrium thermodynamics it is St that governs 
the ongoing macroscopic processes and can be expressed as a measurable quantity via 
Eq.(34). In the absence of external sources (or sinks) the rate of entropy production 
simply describes the relaxation rate; the theoretical maximum entropy itself characterizes 
only the nonequilibrium state from which the system is relaxing to the singular equilibrium 
limit. Further thought leads us to conclude that these interpretations can also be applied 
to ongoing processes. 

The formalism presented here applies to macroscopic systems arbitrarily far from 
equilibrium, although the nonlinear equations provide formidable mathematical barriers 
to any detailed calculations. While the linear approximation is the most fruitful approach, 
even here the covariance functions remain somewhat complicated and resistant to exact 
computation; various attacks have produced some progress, however, in the context of 
linear hydrodynamics. At present it is the formal relations containing covariance functions 
that can prove most useful, in which carefully chosen models of these nonequilibrium 
correlations can play a role similar to that of potential models in equilibrium statistical 
mechanics. Although the present results may lay some groundwork for a complete theory 
of nonequilibrium thermodynamics, there is a great deal of room for expansion and further 
development. 
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